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METHOD AND APPARATUS FOR PROCESSING DIGITAL PDCEL OUTPUT SIGNALS 

BACKGROUND 

1 . Field of the Invention 

The invention relates to processing pixel output signals and, more particularly to 
processing digital pixel output signals. 

2. Background Information 
WW, advances in digital technology, digital cameras, tha, is, cameras that represent the 

mtensrtv of light for a given pixel location of an image in a binary digital signs, forma,, such as 
w„h Ms. are becoming more prevaient. One disadvantage o, the use of such cameras, however 
« «he noise or offset tha, is sometimes encountered wi,h ,he use of digital imaging arrays, such ' 
as arrays ,ha, employ charged couple device sensors or complemen,ary-me.al™de 
sermconductor sensors, hereinafter referred ,o as CCD sensors and CMOS sensors, respectively 
In order ,o address ,his so-ca.led fixed pattern noise ,FPN), one technique tha, has been employed 
is to read a 'dark image" into memory, such as by preventing the sensors of ,he array from being 
exposed to light at a se, of substantially predetermined parameters including, for example 
exposure, temperature and gain factor, and storing the resulting digital pixel output signals. Then 
,ha< stored dark image is subtracted from the image of interest produced using the digit,, imaging' 
array. Typically, in either hardware or software, the digital camera would ensure ,ha, 
^ corresponding pixel output signals for the desired image and the dark image are associated so mat 
„ they may be appropriately subjected or compared. However, the presence of this noise and the 

aPPr ° aCh emPl0Ved " address » "***«•» *Mti°na, complexities into the processing of an 
.J, image ,ha, are not present for cameras tha, store a signal value for the intensity of a pixel in a 
- forma, o,her ,han a binary digi«al signal forma,. A need .herefore exists for techniques ,o address 
,he complexities that digital cameras employing these sensors present . 

25 SUMMARY 

Briefly, in accordance witi, on. embodiment of the invention, a digital camera includes- , 
d,g„al .maging array including , plurality of pixels, and image processing circuitry to process the 
d,g,.a, pixel outpu, signals produced by ,he imaging array. The imaging processing circuitry is 
adap,ed ,o process sa,ura,ed digital pixel outpu. signals differently from non-saturated digital pixel 

30 output signals. 

In accordance with another embodiment of ,he invention, e, least one integrated circuit 
mcludes ,mage processing circuitry. The image processing circuitry is adapted to process digital 
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Pi*e, output signals produced by . digita! imaflin( , array . The 

:r::r sss ~ 

in accordance with ye, one more embodiment of the invention. . method of processing 
d 9, ta, „,«, output signals produced by a di gi ,e, imaging array inciudes process saturated digrta, 
P.xel output signals differently from non-saturated digital pixel output signais. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The subject matter regarded as the invention is particu..rly pointed out and distinctly 
claimed in the concluding portion of the specification. The invention, however, both as to 
organization end method of operation, together with objects, features, and advantages thereof 
may best be understood by reference to the following detailed description when read with the ' 
accompanying drawings in which: 

FIG. 1 is a block diagram illustrating the logical operation of an embodiment of an 
apparatus for processing digital pixel output signals in accordance with the invention; 

FIG. 2 is a flow chart illustrating an embodiment of a method for processing digital pixel 
^ output signals in accordance with the invention; and 

| FIG. 3 is a block diagram illustrating a hardware embodiment of an apparatus for 

m processing digital pixel output signals in accordance with the present invention. 

fgj DETAILED DESCRIPTION 

| In the following detailed description, numerous specific details are set forth in order to 

£ provide a thorough understanding of the invention. However, it will be understood by those 
- skilled in the art that the present invention may be practiced without these specific details In 
25 other instances, well-known methods, procedures, components and circuits have not been 
described in details so as not to obscure the present invention. 

As previously described, digital cameras are becoming more common due, at least in part 
to improvements in digital technology. Therefore, imaging arrays comprising CCD and CMOS 
sensors are being deployed. One aspect, however, associated with the use of such sensors, is the 
presence of fixed pattern noise <FPN). In order to improve the quality of the resulting image 
produced using imaging arrays comprising such sensors, one technique is to acquire a dark image 
store it in memory, such as random access memory (RAM) or dynamic RAM (DRAM) use the 
.maging array to acquire an image of interest, and then subtract or remove the stored dark image 
from the image of interest. Therefore, the FPN is collected by restricting the sensors from being 
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exposed ,o „gh, prefer as Cose as possibie t0 . set „ parameters ^ 

«mper.,u,e and gain facto, employed to produce the image o, Merest, and storing Il a 

tha, m,m,c the conditions for th e ima g e o, interest as closely as possible. Likewise, a 
camera ,nc,ud,n 9 an image processing wil, also inciude digita, camera or imagi ng array circuitry 
adapted to process digital pixe! output signals produced by the imaging array. Therefore the 
c,cu,try wii. synchronic the corresponding digital pixel output si 9 nals produced for the image of 
■meres, and for the dark image so that they may be subtracted from each other or compared „ 
w,ll, of course, be appreciated by one of ordinary skill in the art that this processing may be 
.0 implemented purely in hardware or alternatively in software executing on a hardware platform 
such as induding a processor, such as a microprocessor or drgital signal processor, for example 
In th,s context, the term circuitry is intended to refer to and include any hardware, whether special 
purpose hardware, general purpose hardware with special purpose software, -firmware", or any 
combination thereof. 

1| As previously described, one problem with this approach or technique is that it introduces 

| addit.onal complexity in comparison with approaches that do not employ binary digital signals or 
£ b.ts. For example, when employing binary digital signals of a fixed length, the dynamic range for 
g the intensity of light received by a pixel of the imaging array is inherently limited. Therefore when 
m a pixel is exposed to an intensity of light that exceeds that dynamic range, the image quality is 
2^ affected because the digital pixel output signal becomes saturated or clipped and, therefore, the 
J output signal of the pixel is not an accurate representation of the intensity of the light to which 
u the pixel was exposed. A typical situation in which this might occur, for example, is referred to as 
« "specular reflection". In this context, specular reflection refers to light directly reflected into the 
- sensor from a very intense or bright source, such as the sun, for example. As a result, the 
* dynamic range of the intensity for all the image features is usually well beyond the capability of a 
d.g,tal sensor, leading to signal clipping. Therefore, in a situation where a subset of the pixels of 
the imaging array are saturated or produce clipped pixel output signals, the technique previously 
described of subtracting the dark image may introduce additional noise into the image rather than 
reducing the presence of noise, as is the desired result. 
3 Although both CCD and CMOS sensors produce FPN, with state of the art technology 

CMOS sensors typically exhibit a greater amount. The source of FPN may come from a variety of 
factors, however, typically includes minute defects in the fabrication process of the CMOS sensor. 
Other factors that may produce such noise include the imaging array circuit design or the 
geometry of the array. 
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An embodiment of a digital camera in accordance with the present invention, such as 
.. ustrated by embodiment 300 in RG. 3, includes a digita, imaging array 310 comprising a plurality 
of pixels made up of sensors, such as CCD and CMOS sensors, for example. In addition digital 
camera 300 includes image processing circuitry to process the digita. pixel output signals 
5 produced by the pixels of the imaging array. For example, as illustrated in FIG. 3 in this 

embodiment, the digital pixel output signals each comprising 10 bits are provided at 12 MHz to 
capture interface circuitry 320, although the invention is not limited in scope in this respect This 
raw ,mage data in the form of binary digital signals is then provided to a dark fixed pattern noise 
(DFPN) reduction or removal unit 340. In removaUnit 340, the digital pixel output signals for the 
10 des.red ,mage are compared with the digital-pixel output signals for the dark image. The digital 
Pixel output signals for the dark image are produced using imaging array 310, as previously 
described, and those digital pixel output signals are stored, in this particular embodiment, by a 
RAM table 330 using a data flow controller, such as a direct memory addressing (DMA) controller 
^ 390, and DRAM controller 380. Of course, the invention is not limited in scope to this memory 
1| architecture. When the dark image is produced and stored, the aspect of the removal unit that 
I Perf ° rmS the subtrac *°n « bypassed. Therefore, when the desired image is produced, the dark 
j image is retrieved from memory 330 and the resulting binary digital signals produced by 
| subtracting the dark image from the desired image are then provided to compander look-up table 
yi (LUT) 350. As illustrated in FIG. 3, the binary digital signals are provided to compander LUT 350 
20-^ as 10 bits and are then provided from the compander as 8 bits for the remaining image 
jU processing. In addition, FIG. 3 illustrates that in this embodiment a color tag is also provided. In 
J this particular embodiment, although the invention is not limited in scope in this respect, the color 
JD tag provided designates a color plane for a digital pixel output signal. Any one of a number of 
' possible color spaces may be employed; however, one such color space is the Red-Green-Blue 
25 (RGB) color space. Therefore, three images representing different colors are produced and 

processed rather than a single image and, in this embodiment, a color tag is provided with each 
image. In this embodiment using compander LUTs, three compander LUTs, one corresponding to 
each color, are employed. If will, of course, be appreciated that the invention is not limited to 
producing color images or to the RGB color space. 
0 One aspect of this embodiment of the digital camera in accordance with the present 

invention is that the imaging array circuitry is adapted to process saturated digital pixel output 
signals differently from non-saturated digital pixel output signals. This aspect of this particular 
embodiment is illustrated in greater detail in FIG. 1, although, again, the invention is not limited in 
scope to this particular embodiment. As illustrated in FIG. 1, the image is first captured or 
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acouired as illustrated by block 1 10. The uncorrected digital pixe, output signals are then 
pr v ( ded to saturation detect and contro, ,ogic 1 20. if a particu.ar dioital pixel output sfcna, ls not 
saturated, the captured uncorrected signal value is provided to node 130 so that the 
corresponding signa, vaiue of the dark image may be subtracted. However, this subtraction does 

ir;: : sat ; atjon detect and contro ' ,ogic indicates that the — «*- *- out: 

-goal of the des.red ,mage is saturated. ,„ this embodiment, this saturation detect and contro. 
log* .. included in dark fixed pattern noise removal unit 340. As previously described, the dark 
.mage ,s acquired and stored prior to producing and storing the desired image. Therefore as 
illustrated in FIG. 1, DRAM controller 160 and DMA 170, respectively, provide the stored digital 
10 p.xel output signals and the corresponding address in this particular embodiment. Of course the 
mvention is not limited in scope in this respect. Therefore, in this embodiment, line buffer 150 
then provides the corresponding digital pixel output signals of the dark image to be processed with 
the uncorrected digital pixel output signals of the desired image that was captured as indicated by 
^ block 1 10. Once the subtraction has occurred, the corrected digital pixel output signals are then 
ijg provided to compander LUT 1 40, as illustrated in FIG. 1 . 

In this particular embodiment, although the invention is not restricted in scope in this 
respect, as previously described, 1 0 binary digital signals or bits are used to represent an intensity 
O level. Therefore, a digital pixel output signal is clipped or saturated when 10 ones or a 

hexadecimal signal value "3FF" is detected. Therefore, in one particular embodiment of a digital 
camera or digital camera circuitry in accordance with the present invention, when the signal value 
5 " 3FF " iS det6Cted 98 the digitaI P* el out P ut ^gnal for a particular pixel of the desired image, 
u instead of subtracting the corresponding digital pixel output signal of the dark image for that pixel 
J tHe subtraction is bYPa—d or turned "off/ as indicated, for example, in FIG. 1 . The saturated 
s«gnal value is provided to compander LUT 140 in FIG. 1 or 350 in FIG. 3. 

This embodiment is also illustrated by the flow chart of FIG. 2. In this embodiment, an 
uncorrected digital pixel output signal is received by the removal unit at block 210. At block 220, 
a decision is made regarding whether the dark fixed pattern noise reduction or removal feature is ' 
"turned on." This feature would not be turned on, for example, where the dark image is being 
acquired and stored. In this case, as illustrated in FIG. 2 at block 270, these processed binary 
digital signals are passed to the next stage, which in this particu.ar embodiment results in applying 
these signals to the compander. However, if this feature is "turned on," and a saturation special 
case has been encountered, the circuitry detects whether the signal value equals "3FF" in this 
particular embodiment, as illustrated by block 240. If not, then the corresponding signal value 
from the dark image is subtracted as indicated by block 260. However, if this binary digital signal 
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vaiue is detected, i.e., -3FF" in this embodiment, then instead o, performing the subtraction th , 
parser bi nar y «#* s, a „a, vaiu. is passed thresh as ,h. oinary W s^nai 
indicated by block 250. ' as 

In yet another embodiment of a digital camera in accordance with the present invention 
he saturated digita, pixel output signa.s are processed differently from the non-saturated pixel' 
d ' 9 ' ta ' ° UtPUt SI ' 9na,S; h0W6Ver ' this diffe ™< Passing includes more than bypassing the 
subtraction of the corresponding digital pixel output signa.s of the dark image. For example in 
one embodiment, an average dark fixed pattern noise level may be subtracted from all the ' 
saturated binary digita. signal va.ues. This approach provides severe, advantages. Due to the 
nature of the fixed pattern noise, typical.y the dark image signal va.ues that would be subtracted 
in the absence of an embodiment for the present invention would be non-uniform and, therefore 
would provide an uneven or patchy appearance to the image if subtracted from saturated signal ' 
va.ues. More specifica.,y, the human eye wou.d expect the image in this region to be uniform or 
clipped and by subtracting the dark image signal values, noise would thereby be introduced 
Therefore, it is desirab.e to subtract an average value rather subtract the particular signal vaiues of 
the dark ,mage from the corresponding saturated signal values. Furthermore, where saturated 
signal values are encountered, the relative intensity of those saturated va.ues in comparison with 
the non-saturated values would be greater than desired if the dark image signal va.ues were 
subtracted from only the non-saturated signal va.ues. This may undesirably affect the appearance 
2* m of the .mage to an observer. Therefore, it would also be desirable if a va.ue could be subtracted 
nj from the saturated signal values to reduce this relative intensity. This subtraction reduces or 
y aV °' dS 3 W9Ste ° f the d * namic ™9 e a ^°le that comes from a high relative intensity. 

Any one of a number of techniques to obtain the average signal value to be subtracted 
from the saturated signal values may be employed. In this particular embodiment, the signal 
values of the dark image are sampled to produce a mean and standard deviation and thereby 
approximate the level of the fixed pattern noise. Of course, a variety of different sampling 
techniques may be employed in different embodiments. For example, it may be desirable to 
sample only in the saturated regions of the desired image. Alternatively, it may be desirable to 
examine the standard deviation and, depending upon the value of the standard deviation, 
subsample regions of the image in order to reduce the standard deviation. Likewise, trends in the 
no.se may be observed by processing selected portions of the dark image, such as where, for 
example, the mean and/or deviation may be larger in one portion of the image than in another. 

Likewise, in another embodiment of the invention, subtracting the value determined as the 
average dark fixed pattern .eve. signal va.ue may be incorporated in the processing imp.emented 
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2 T ' S ° ^ ^ SUbtraCti °" °~ -V be combined with other 
operations ,such as a compander operation and an operation to adiust for the differences lean 
coior response of the sensor and the human eye, referred to as a gamma operation. More 
specially, in one embodiment although the invention is not iimited in scope in this respect 
the compander operation, ten bits are received and from this eight bits are produced This is' 
substant.ally non-iinear operation impiemented using a look-up table. A reason this particular 
operation is substantially non-linear is because this operation is intended to ref.ect or correspond 
to the response of the human eye to light. Therefore, in this particular embodiment, the values for 
the LUT are arrived at by beginning with an analytical model of this response and then adjusting it 
10 emp.ncally. Likewise, a gamma operation is also employed to adjust the color response of the 
resulting image. More specifically, in this particular embodiment, a color filter array (CFA) is 
included inherently in the imaging array sensors, although this may be accomplished 
and the invention is not restricted in scope in this respect or even to employing a CFA. Therefore 
^ a gamma operation is employed in order to invert or remove the effect this color filter array has 
1J had on arriving at calibrating color in the resulting image. Also, in this particular embodiment as 
| PreVi0US ' y d6SCribed ' the si 9 nal va,ue determined to be or represent the level of the dark image 
f Pattem fiX6d n ° iSe maV lik6wlse be subtracted. Furthermore, in this embodiment, because the 
| value of a clipped or saturated digital pixel output signal value is known, subtraction may be 
Ul implemented by merely storing the signal value to be subtracted. Of course, the invention is not 
20^ restncted in scope in this respect. As is wel.-known, the three operations previously described 
m may be combined and implemented using a single look-up table. Although the precise steps to 
J accomplish this are not discussed here, such techniques are clearly within the level of skill of one 
jO of ordinary skill in the art to which this invention pertains. Again, as previously described, 
- different look-up tables are employed for each color plane in this embodiment, although the 
25 invention is not limited in scope in this respect. 

In yet another embodiment of a digital camera in accordance with the present invention 
imaging processing circuitry may include the capability to identify saturation regions of the image 
before processing the saturated digital pixel output signals differently than the non-saturated 
digital pixel output signals. More particularly, in one embodiment, for example, the removal unit 
SO may be adapted to check the output signals of bordering pixels for a pixel producing a saturated 
output signal to verify that all the immediately adjacent pixel output signals are also saturated. In 
one embodiment, if not all are saturated, then this saturated signal value may not be processed 
differently. Alternatively, if a particular number of signal values in the surrounding region are 
detected to be saturated, then the output signals for pixels that are in that region and saturated 
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may be processed different Other embodiments to determine end/or designate a reclon , 
saturated digital pixel output signal values may be employed Likewise I 
o^i gna, value is ,den.ied. .a image ^'J^^TZ 

27 e « • - '" • - ~ signal values. For exampie. where the 
5 sja, value ,s an .solated saturated signa, vaiue, no different processing in comparison SI 
saturated p,xe. output signais might be empioyed. Again, afcemative approaches may be 

employed. 

Of course, as previous^ indicated, embodiments may be implemented a variety of ways 
For example, as previously described, the processing may be "hardwired." Alternatively as 
10 previous* described, a processor, such as a DSP or microprocessor, may be loaded with software 
that, when executing on the processor, performs the desired operations. Alternatively an 
embodiment may be implemented in firmware as we.l. Although the invention is not limited in 
scope ,n this respect, it is typically more desirable to imp.ement embodiments in ways that result 

Q h ' 9her SPeed Pr ° CeSSing dUe t0 the re,ative| V high number of binary digital signal values or bits 

IS to be processed. 

1 WhHe Certain features of the invention "ave been illustrated and described herein, many 

£ m ° d,f iCatl '° nS ' Substitutions ' chan ** «nd equivalents will now occur to those skilled in the art It 

g «s, therefore, to be understood that the appended claims are intended to cover all such 

m modifications and changes as fall within the true spirit of the invention. 
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